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Abstract 

This study aimed at evaluating the cognitive levels of the questions following the reading texts of Master Class 
textbook. A checklist based on Bloom’s Taxonomy was the instalment used to categorize the cognitive levels of 
these questions. The researchers used proper statistics to rank the cognitive levels of the comprehension 
questions. The results showed that the author of Master Class emphasized the cognitive level of Comprehension 
having 52% of the questions, which was much more than the expected frequency, while wrote only 3.7% and 6% 
of the questions on the cognitive levels of Knowledge and Application respectively. The frequency of questions 
on the cognitive levels of Evaluation and Analysis were much closer to the expected frequencies. The results 
indicated that about 40% of the textbook’s questions emphasized higher-order thinking skills, which goes with 
the requirements of the revised curriculum. Evaluating and choosing a good textbook that goes with the goals of 
the curriculum is recommended. Such a study would shed light upon the role of textbooks in developing 
cognitive skills among Arab students. 

Keywords: Bloom’s taxonomy, questions, revised curriculum, textbooks’ analysis 

1. Introduction 

Knowledge, as an outcome of education, is no longer believed to be sufficient to create the kind of citizens 
needed to effectively cope with the social, economical and technological changes in the world. In the last few 
decades, there has been an intense call to raise the level of educational achievements. In contrast with the 
previous generations, Wagner (2008, p. 21) asserts that the global information economy now requires greater 
breadth and depth of skills in making meaning. Students accordingly “need to master seven survival skills: 
critical thinking and problem solving, collaboration and leadership..., effective oral and written communication, 
accessing and analyzing information, curiosity and imagination.” Shavinia (cited in Yamin, 2013) claims that 
citizens who think critically and creatively are “guarantees of political stability, economic growth, scientific and 
cultural enrichment, psychological health, and the general prosperity of any society in the 21 st century” (abstract). 
As a result, more attention has been given all over the world to the importance of developing students’ HOTS 
(Gallaghera, Rosemary, & Zohar, 2012; Lin, 2005; Yamin, 2013; Zohar, 2007). 

Many nations have become more concerned with the incorporation of the cognitive skills within their national 
curricula (Craft, 2005; Shaheen, 2010; Gallagher et al., 2012). Curriculum reforms have been carried out and 
higher-order thinking skills (HOTS) have been included in educational policies in many countries (Shaheen, 
2010; Lin, 2011). “Like other countries, [in 2007], Israel has had its share of projects that see the implementation 
of inquiry and higher order thinking in schools as their main goal” (Gallagher et al., 2012, p. 139). These 
projects aim at enabling students to grasp a deep understanding of what they are learning and be more critical 
and creative instead of merely recalling information. 

1.1 Infusing HOTS in the English Curriculum 

Since the English language can play a vital role in enabling students to function successfully in this highly 
technical society and fast-paced change in the world, the English Curriculum Committee’s and the English 
Inspectorate decided to infuse HOTS to English teaching through teaching literature. Accordingly, since 2008, 
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high school teachers have been required to teach a number of literary pieces (short stories, articles, poems, plays 
and novels) while infusing and emphasizing the following six thinking skills: comparing and contrasting; 
distinguishing different perspectives; explaining cause and effect; explaining patterns; inferring; and problem 
solving. Therefore, teachers must explicitly teach thinking skills through focused instruction and ask the right 
instructional questions about the literary texts taught to enhance students’ thinking. 

Though it is easier to teach HOTS while teaching literature, it is not enough to concentrate on these skills only 
when teaching literary texts. HOTS should be emphasized while teaching any kind of texts to improve 
understanding. 

Reading comprehension is the core of the teaching/learning process in all disciplines. Students cannot understand 
history, biology, social science or math if they do not read proficiently and critically. As a result, textbooks must 
emphasize reading comprehension strategies and guide students to read critically while reading in the language 
lessons. When students acquire these skills by practicing them on various texts, they will be able to transfer them 
while reading in different disciplines and later while reading outside the school scope, which is the main goal of 
teaching HOTS. Sidek (2010, p. 83) asserts that “secondary students may not be prepared for the more difficult 
cognitive demands of reading tasks in EFL at the tertiary level” if they are only exposed to reading tasks that 
require low cognitive demands. Enabulele (2011, p. 7), on the other hand, claims that “students who are 
proficient in reading have a greater chance of success in the job market.” 

1.2 How Are the Levels of Comprehension Developed? 

Reading is the basic means to get knowledge. Unfortunately, however, while reading, many students focus more 
on word accuracy rather than comprehension and in most cases they can only understand what they read at the 
literal level. They can infer what the words say superficially but find it difficult to read between the lines to get 
the underlying meaning and intentions of the writer (Fisher, 2005). These readers “possess inefficient reading 

strategies and use them inflexibly. [They, accordingly,] need to be shown how and when to apply a small 

repertoire of comprehension strategies” (Department of Education and Training, 2010, p. 2). In order to provide 
the students with a richer, meaningful reading experience, they have to be moved from the literal or concrete 
level of thinking to higher levels in which they do something with the facts they get from the texts they read 
(Curriculum Associates). 

The thinking process that is operated by students is essential for mental health, high achievements, and 
professional success. As a result, students should be encouraged to give opinions about what they read, to 
analyze materials, to form creative ideas, to evaluate, to compare and contrast different choices and to relate 
what they read with their own experiences. With sufficient explicit instruction of the reading strategies, 
noticeable gains in comprehension will be achieved by students. When the students get to the level in which they 
use the reading strategies simultaneously, they will know when to apply each strategy while reading any text, 
and only then they “can read a variety of materials with ease and interest, can read for varying purposes, and can 
read with comprehension even when the material is neither easy to understand nor intrinsically interesting” 
(Snow, 2002, p. xiii). 

Acquiring these strategies is crucial for students’ success in all disciplines. Several taxonomies have been 
proposed as a result of numerous studies in the area of questions and objectives, such as those proposed by 
Guilford, Weaver, and Bloom. These taxonomies embody the levels of three educational objectives by which 
activities are posed—cognitive, psychomotor, and affective. The cognitive taxonomy, which was proposed by 
Bloom (1956), is the most commonly used in the area of education. Krathwohl (2002) states that 

Bloom saw the original Taxonomy as more than a measurement tool. He believed it could serve as means for 
determining the congruence of educational objectives, activities, and assessments in a unit, course, or 
curriculum; and panorama of the range of educational possibilities against which the limited breadth and depth 
of any particular educational course or curriculum could be contrasted” (p. 212). 

The six levels within the cognitive domain are divided into two levels of thinking skills: 

• Lower-Order Thinking Skills (LOTS): Knowledge, Comprehension and Application 

• Higher-Order Thinking Skills (HOTS): Analysis, Synthesis and Evaluation. (Bloom, Englehart, Furst, Hill, & 
Krathwohl, 1956; Truschel, 2010) 

Many researchers who have studied how levels of comprehension affect reading comprehension have taken 
Bloom’s Cognitive Taxonomy as a frame for their work. Curriculum designers, book writers and teachers also 
consider using it as a “tool to ensure appropriate coverage of a variety of types of cognitive demands made on 
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students [with a] range of learning processes [that extend] from the common lower tasks to include higher level 
cognition” (Surjosuseno & Watts, 1999, p. 7). 

1.3 The Role of English Textbooks in Developing Students ’ Higher-Order Thinking Skills 

It is widely acknowledged that textbooks exert a powerful and precious influence on education. English language 
textbooks, however, evoke different responses among English teachers. Some believe they are “masses of 
rubbish skillfully marketed” (Brumfit, cited in Sheldon, 1988, p. 237) and prefer not to use them, while others 
consider them “necessary evils” (Sheldon, 1988, p. 237), which because of the “sheer labor-intensiveness of 
developing classroom materials, the pressures of heavy timetable, and the highly restrictive nature of most 
teaching situations, [they find themselves forced to] choose a book which only approximates to the needs of the 
local context” (Sheldon, 1988, p. 238). 

Their choice is usually done in haste, having no systematic applied criteria. This makes them unhappy about 
their compromise and, accordingly, find themselves, instead of dealing with the textbook as a route map for the 
whole course, go in the maze of choosing the parts from the textbook that adequately fulfill their teaching 
objectives, modifying the parts that do not suit their students well, and bringing teacher-generated materials that 
help in imposing the components they feel the textbook lacks. 

One of the basic components of textbooks is the text. Shulman (1987, p. 15) asserts that “[t]hough most teaching 
begins with some sort of text, and the learning of that text can be worthy end in itself, we should not lose sight of 
the fact that the text is often vehicle for achieving other educational purposes.” In order to help students achieve 
the educational purposes intended, questions, which are the second basic component of the textbooks, are needed. 
Good questions and well-built activities that follow each text in a textbook should be written in a way that helps 
attaining the educational objectives and lead students to think critically. 

Research has revealed that “higher order questions may have a somewhat broader general facilitative effect than 
factual adjunct questions” (Hamaker, 1986, p. 237). Andre (1987, p. 81) also claims that “higher level adjunct 
questions facilitate the learning of factual information from text and increase the amount of attention readers 
devote to processing text.” Besides, Armbruster and Ostertag (1989, p. 3) state that when students are asked 
questions on a certain text they “form expectations based on the type of question they receive, and these 
expectations affect learning from reading subsequent material. ...[Students’ interaction with questions directly 
influences future learning outcomes.” This emphasizes the role questions play to promote higher order 
processing of the text. 

English textbooks are usually full of questions that come either at the beginning or at the end of each section, 
lesson or chapter; unfortunately however, research has shown that most textbooks do not contain materials, nor 
do they include questions that require critical thinking and meta-cognitive processes (Collins, Brown, & 
Newman, 1989). Most textbooks questions, as research indicates, emphasize the lower-order cognitive levels 
(Cotton, 1991; Ighbaria, 2013; Riazi & Mosalaejad, 2010). Accordingly, the cognitive levels of the textbook 
questions should be one of basic criteria to be used to evaluate textbooks. 

The situation in Israel is similar to the situation in some other countries where there is no fixed textbook from the 
Ministry of Education for each level in the whole country. Teachers usually have a variety of textbooks approved 
by the Ministry of Education published by two main publishing companies for each grade and proficiency level. 
The teachers in each school consequently choose a textbook either because it is highly recommended by one of 
the publishing companies, or because other teachers recommend it. 

Many English teachers do not thoroughly evaluate the textbook they choose to adopt, either because they feel it 
is tiring and time-consuming process, or because, in many cases, they are not qualified enough to do so. They do 
not have the competence and the experience to evaluate a book before using it. They claim that the textbook is 
only a tool or a teaching aid that they can manipulate to serve their educational and learning objectives. 

By doing so, however, they ignore the fact that textbook evaluation either before, while or after using the 
textbook, helps teachers in their personal and professional development. It is, as Hutchinson (1987, p. 38) says, 
“a two-way process which enables teachers not just to select a textbook, but also to develop their awareness of 
their own teaching/ learning situation.” It makes teachers acquire and develop understanding regarding the nature 
of textbook material in a general manner, rather than impressionistic assessment. It helps them choose the 
textbook that is compatible with their teaching objectives, and suits the level and needs of their students, which 
ultimately ensures high quality teaching. 

As a result, the researcher saw it important to analyze one of the textbooks used to teach English for high 
proficiency level students to find out to what extent the recent trend of developing students' HOTS was reflected 


102 




www.ccsenet.org/elt 


English Language Teaching 


Vol. 8, No. 5; 2015 


in the questioning practices of the textbooks, and accordingly help teachers choose the textbooks that adequately 
facilitate the teaching process and prepare students for high cognitive demands of reading tasks in EFL at the 
tertiary level. 

1.4 Review of Related Studies 

A number of studies have been conducted to investigate the type and the cognitive level of the textbooks’ 
questions in different subjects and different countries around the world using Bloom’s taxonomy as a guide for 
categorizing the questions. 

Karns, Burton, and Martin (1993), for example, studied six principles of economic texts and the instructor’s 
manuals that accompanied them using content analysis. The aim of their study was to determine if the questions 
contained in the instructor’s manual really measured the level of achievement of stated course objectives. 

The results revealed that though most authors placed few learning objectives on the first three levels of 
knowledge, comprehension and application, the questions on these three levels were significantly given more 
importance. On the other hand, the other three levels, which related to the higher-order thinking levels in 
Bloom’s Taxonomy, were not addressed as a learning level neither as objectives nor as questions by most the 
texts’ authors. Even when listed as part of the learning objectives, the authors failed to provide questions on 
them. 

Ibrahim (1998) analyzed 6 th -grade history book questions according to the cognitive domain in Bloom’s 
taxonomy in Iraq. Out of the 87 questions that he found in the book, 72% of the questions were asked at the 
cognitive level of knowledge. 25.4% of the total of the questions were asked at the comprehension level and the 
evaluation questions received 2.2%, the three other levels - application, analysis and synthesis scored 0%. 

Riazi and Mosalaejad (2010) investigated which levels of Bloom’s taxonomy were more focused on in Iranian 
high school and pre-university textbooks. They evaluated three senior high school and one pre-university 
textbooks using Bloom’s taxonomy.The results obtained showed that the lower-order cognitive skills were more 
dominant having learning objectives on comprehension, application and knowledge more frequently. Objectives 
on the level of evaluation were the least in the four English textbooks. Analysis and synthesis came in between 
in all the books. 

Razmjoo and Kazempourfard (2012) analyzed the activities and the exercises for three units of each of the four 
coursebooks of the Interchange series using the six levels of Bloom’s Revised Taxonomy. The researchers 
utilized a coding scheme to codify, classify and analyze the exercises and activities of the coursebooks. The 
results showed that lower order cognitive skills were most frequent in Interchange coursebooks. Remembering, 
which is the lowest cell in Bloom’s Revised Taxonomy, was the most frequent code followed by applying in the 
four books. Understanding metacognitive knowledge and evaluating cognitive knowledge were found to have 
the least frequent codes with 0% of distribution. 

Ighbaria (2013) analyzed the six units of the 9th grade English textbook Horizons. His main objective of the 
study was to assess the importance of textbooks in developing students’ thinking. He chose the WH-questions as 
the unit for analysis believing that questions are important for examining students’ understanding for the taught 
material, and that through questions, students’ thinking skills can be developed. The results revealed that out of 
the 381 WH-questions that the six units included, the percentages ranged from 29.66%-2.36% which related to 
the cognitive levels of comprehension and evaluation respectively. These results confirm the results that 
appeared in previous studies. The outstanding finding was that the analysis level appeared at a percentage of 
23.36%, which is nearly equivalent to the knowledge level. 

1.5 Statement of the Problem 

Textbooks writers and publishing companies are working on writing new textbooks that meet the objectives of 
the revised curriculum. No new textbook, however, has currently been published for high school that meets these 
requirements, accordingly high school teachers are still using textbooks published at least six years ago. The 
problem that English teachers face is that there are several books for each level offered by the different 
publishing companies that are approved by the Ministry of Education. Teachers have difficulty choosing the 
book suitable for each level, which adequately fulfills the goals set by the curriculum, and, in the same time, 
helps them in developing students' thinking levels. 

The researcher chose Master Class textbook, which is written for proficiency level, stage one students, to 
investigate the cognitive levels of the questions in the Mastering Reading sections in order to find out the extent 
to which they lead students towards levels that demand higher thinking skills as the revised curriculum entails. 
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Accordingly, the study addresses the following question: 

• What are the cognitive levels of the questions that follow the reading texts in Master Class textbook? 

1.6 Significance of the Study 

The researchers think that this study is the second of its kind that investigates the cognitive level of the questions 
in high school English textbooks in Israel. Several studies were conducted in the last few decades to examine the 
type and cognitive levels of the questions that appear in textbooks in different countries and in different subjects 
(Armbruster & Ostertag, 1989; Hampton, Krentler, & Martin, 1993; Ibrahim, 1998; Karns, Burton, & Martin, 
1993), while very few studies dealt with the type and the cognitive level of the questions given in English 
textbooks (Razmjoo & Kazempourfard, 2012; Riaz & Mosalaejad, 2010). Only one study, lghbaria (2013), 
investigated the type and the cognitive level of the questions in the textbooks used in Israel. In consistence with 
all the other studies, the results of the last study revealed that the authors of the textbooks tended to emphasize 
lower cognitive levels in the textbook questions. 

The results of this study will therefore contribute to all those involved in the educational practice of English 
teaching in Israel, the textbook authors as well as the Ministry of Education. The results of the study will also 
raise their awareness of the cognitive levels of the questions included in these textbooks. In addition, the results 
will alert teachers to the importance of evaluating the textbooks offered by publishing companies in order to 
choose the ones that better help them achieve the curriculum objectives. Otherwise, they must modify their 
practices and materials in a way that makes them achieve higher levels of teaching/learning objectives. 

1 .7 Definitions of Relevant Terms 

Questions are any of the following: a R7?-question, a Yes/No question, a Multiple Choice question. Complete the 
sentence or the Chart, a Statement question, a Request question, or an activity. 

Master Class textbook . Master Class is a course book for the tenth grade high school students at Proficiency 
Level, Stage 1. It was written by Edna Assis and was published in 2008 by Eric Cohen Books Ltd. Master Class 
is divided into seven sections; two Mastering Reading sections, two Mastering words sections, one Mastering 
Language section, one Mastering Listening section and one Mastering Writing section. The Mastering Reading 
sections include different kinds of texts (interview, article, newspaper editorial, personal account, review, 
website and timeline), preceded and followed by several types of reading comprehension questions. 

2. Methodology 

2.1 Instrument of the Study 

For the purpose of this study a checklist was used to record and tally the cognitive levels of the questions 
collected from the Mastering Reading sections of Master Class textbook. 

The researchers built a checklist based on Bloom’s Taxonomy of the cognitive domain. The checklist is 
composed of a table with eight columns. The first column contained the serial number of the question; the 
second contained the question, while each of the following six columns contained one of the six cognitive levels 
of Bloom’s Taxonomy placed in sequence from low to high. 

The checklist was combined with explanatory sheets that contained two tables: a table in which a definition for 
each of the six cognitive levels and examples on the verbs used for each level, and a table with example 
questions and activities on each level. The tables were added to make it easier for the researchers decide the 
cognitive level of each question coded. 

2.1.1 Validity of the Checklist 

Bloom’s taxonomy was used and determined to be valid in various previous studies (lghbaria, 2013; Razmjoo & 
Kazempourfard, 2012; Riazi & Mosalaejad, 2010). It was also accepted among the educational community as a 
means of categorizing and “determining the congruence of educational objectives, activities, and assessments in 
a unit” Krathwohl (2002, p. 212). Accordingly, the checklist, being directly derived from Bloom’s Taxonomy, 
was considered valid. 

To further validate the checklist, its concepts were first defined accurately, depending on the definitions given in 
the literature about Blooms taxonomy. The checklist was displayed to a panel of five experts to examine the 
definitions of the levels according to the skills and questions demonstrated for each level. There was a consensus 
among them regarding these categories and the definitions. The instrument also appeared appropriate for the 
study. 
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2.1.2 Reliability of the Checklist 
Inter-Rater Reliability 

Reliability of the checklist depends on the raters’ utilization of the instrument. In order to establish reliability of 
the checklist, the researchers collected, coded and categorized all the questions following the reading texts of the 
first and the second units of another textbook called Results for 4 points. 

Using Holsti’s equation to calculate the reliability coefficient (Holsti, 1969), the researchers found that the 
agreement coefficient was 91.3%. 

2.2 Data Collection 

The researchers obtained two copies of Master Class textbook. Each used the the book separately to code the 
questions depending on the instructions given earlier. 

The first results of coding the questions were identical: both the researcher and the second analyst found 135 
questions. The questions were then categorized according to Bloom’s Taxonomy using the checklist. 

2.3 Data Analysis 

To analyze the cognitive level of the questions of the reading comprehension sections in Master Class textbook 
multiple steps were used. 

From the coded questions taken from the textbook, the frequencies for each level of cognition in each unit were 
firstcalculated. The totals for each cognitive level in the six units were summed to derive a total for each level of 
cognition. The total for each level was divided by the grand total of the questions in the six units to arrive at the 
percentage for each cognitive level within the six units. Chi-square goodness-of-fit test was then used. The 
expected frequency and the residual for each thinking skill were calculated, and then the standardized residual 
was computed. The aim was to rank the cognitive levels of the reading comprehension questions according to 
their frequencies. 

3. Results and Interpretations 

Table 1 presents the cognitive levels of the reading comprehension questions according to their standardized 
residual values in a descending order. 


Table 1. Frequencies, percentages and chi-square goodness-of-fit values for the cognitive levels of the questions 
arranged in a descending order according to the standardized residual values 


Rank 

Level 

ID 

Question 

Observed 

N 

% 

Expected 

N 

Residual 

x 2 

df 

Sig. 

Standardized 

Residual 

1 

2 

Comprehension 

70 

51.8 

22.5 

47.5 

100.278 

1 

0.000 

10.014 

2 

6 

Evaluation 

21 

15.5 

22.5 

-1.5 

0.100 

1 

0.752 

-0.316 

3 

4 

Analysis 

20 

14.8 

22.5 

-2.5 

0.278 

1 

0.598 

-0.527 

4 

5 

Synthesis 

11 

8.1 

22.5 

-11.5 

5.878 

1 

0.015 

-2.424 

5 

1 

Knowledge 

8 

5.9 

22.5 

-14.5 

9.344 

1 

0.002 

-3.057 

6 

3 

Application 

5 

3.7 

22.5 

-17.5 

13.611 

1 

0.000 

-3.689 

Total 



135 

100 

135 

0 

129.489 

5 

0.000 

0 


Table 1 shows that the total number of the questions that were categorized according to Bloom’s Taxonomy was 
135. The frequencies of the questions of the six cognitive levels range from 5-70, while the percentages range 
from 3.7% for the cognitive level of application to 51.9% for the comprehension level. The standardized residual 
values indicate that the observed frequency of the questions on the comprehension level is significantly higher 
than the expected frequency for that level. 

The thinking skill that appears most frequently after comprehension is evaluation with 21 questions (15.6%) 
which is the same as that of the expected frequency for these questions according to the standardize residual 
value. Analysis comes third with 20 questions (14.8%). This number also approximately equals the number of 
the expected frequency for this skill. The fourth skill is synthesis with 11 questions (8.1%). The number of 
questions on this skill is significantly lower than the expected frequency of these questions. There are 8 
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questions (5.9%) of knowledge, and 5 questions (3.7%) of application. The standardized residual values for these 
two skills indicate that the expected frequencies of the questions for both skills are significantly higher than the 
observed frequencies. 

These results show that Master Class textbook is successful in adopting the two higher thinking skills of 
evaluation and Analysis. There is shortage, however, in the number of the synthesis questions, the third higher 
thinking skill. On the other hand, the number of questions on the cognitive level of comprehension is too high. 

4. Discussion, Conclusions and Recommendations 

4.1 Discussion 

Scholars who examined the effect of the questions on foreign language learning indicated that the level of the 
cognitive demand of language tasks is an important factor in language mastery (e.g., Igbaria, 2013; Lin, 2009, 
2010; Sidek, 2010). In order to achieve the required information needed for academic success and for life, the 
cognitive demands of the reading tasks given for students should be of various levels. More emphasis, however, 
should be put on tasks that need high cognitive demands (Ministry of Education, 2011; Zohar, 2008). 

The results showed that Master Class textbook included reading comprehension questions that required both 
lower level and higher level cognitive demands. They, however, showed that the cognitive levels of the questions 
were not fairly varied. First, 40% of the questions in Master Class needed HOTS while 60% needed lower-order 
thinking kills (LOTS). Second, while there were knowledge and application questions, they were relatively few 
when compared with the frequency of comprehension questions. There were, for example, 70 comprehension 
questions, while there were only five application questions. 

Having more LOTS than HOTS questions should not affect the judgment on the textbook for several reasons. 
First, Bloom (1956) emphasized the importance of offering lower level information to students as a basis to 
move to upper levels of cognition. Armbmster and Ostertag (1989, p. 2) also claimed that “no one suggests that 
all questions should be high level”. They added that “the use of factual questions can be defended on the grounds 
that students need to know certain basic information before they can engage in higher order thinking”. 

Second, taking a closer look at the number and the nature of the questions that require lower-order thinking skills, 
the researchers believe that the author of the textbook was successful in her choices. The cognitive level of 
knowledge appears only 5.9% of the time in all the units. This result contradicts with the results of most of the 
previous studies (Igbaria, 2013; Razmjoo & Kazempourfardm, 2012; Riazi & Mosalaejad, 2010), in which more 
questions or even most of the questions in the textbooks were of the knowledge type. 

Moreover, other than having only eight Knowledge questions out of the 135 questions, which is a positive 
change, such questions are presented in a way that requires the students to think and interact, and not just to 
recall information. On page 84, for instance, there is the following question,” You are going to read a review of a 
movie about global warming. Share what you know about the subject with the class. The following words from 
the review may help you.” On page 92, there is another question that asks students to recall information they 
know about different topics, yet the question asks them to recall the information by asking them to expect what 
they will find when they click on several internet links, so it connects them with technology. 

The application cognitive level received the lowest percentage in the textbook analysis having only five 
questions/activities (3.7%) in the twelve reading sections of the textbook. These results also contradict with some 
of the results of the previous studies (Karns et ah, 1993; Razmjoo & Kazempourfard, 2012; Riazi & Mosalaejad, 
2010), where the application questions appeared more frequently. 

The researchers believe that this number of questions is considered logical in these sections since they focus on 
receptive and not on productive skills, the access to information domain and not the presentation domain. 
Questions that belong to the cognitive level of application are expected more in sections like Mastering language 
or Mastering Writing. 

The third cognitive level related to the lower-order thinking skills is comprehension. The results show that there 
are 70 (51.8%) comprehension questions. Having such a high frequency of these questions is also quite 
reasonable in this study as the data were collected from sections that focused mainly on the domain of access to 
information and where the questions in these sections should emphasize students’ comprehension. 

There is no doubt that the author of Master Class tried to expose students to different types of questions to make 
the texts more comprehensible. There are, for example, several !!7?-questions like “Why do supertasters taste 
things more intensely than other people? (line 19)”; Yes/ No questions as “Choose the correct answer Yes or No. 
Most people continued to use black and white film long after the invention of color film. Yes/ No,” and Requests 
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“Name two other results of global warming according to lines 49-60”, to mention just a few. The textbook also 
contains several activities in which the students are asked to work alone, in pairs or in groups to do diverse tasks. 

However, instead of having questions that needed higher levels of processing, the textbook author, presented 
questions that need in most cases short and direct answers. Most of her questions were ‘ Display ’ or 
‘close-ended’ questions which have only one right response that could easily be detected from the text. 

An example of such questions is: “What are two advantages of being a Super taster? 

a) Supertaster do not usually_because they_. 

b) They also do not usually_because they_(p. 17) 

In fact, there are some ‘ open-ended ’ questions, yet the kind of questions that she wrote did not need “complex 
inferences, such as pulling together information from several locations in the text,” (Armbruster & Ostertag, 
1989, p. 8), which require higher cognitive demands. There is, for instance, this question on page 86, “Name two 
other results of global warming, according to lines 49-60”, and on page 109, we have this question: “The Navajo 
language was difficult for non-Navajos to learn. Give ONE reason of this, (lines 15-19).” The answers for these 
questions are easily detected from the text. 

The third reason for which the judgment of the textbook should not be negatively affected by the results is the 
frequency of the questions that required the higher cognitive levels. The number of the questions that required 
the cognitive levels of analysis and evaluation is comparatively high. They appeared at the frequency of 21 and 
20 and the percentages of 15.5% and 14.8% respectively, which is quite a good percentage not only compared 
with the results of the previous studies but also with the kind of questions that are usually asked after any reading 
comprehension text in a foreign language where the main goal is usually comprehension and not developing 
thinking skills. There is, for example, an analysis question on page 45, “Which of the following words describe 
how you felt as you read this editorial? Explain,” As for the evaluation level, we can see the following question 
“Imagine that you’ve been in a Europeans city and wanted to buy a newspaper, but there is no one to take the 
money—just an “honesty box” to put it in. Would you pay for the newspaper or not?” and on page 26, “Yair 
advises parents to keep track of their kids. What do you think of his advice? Should parents check up on their 
teenagers? “ 

Synthesis questions appeard at a frequency of 11 and a percentage of 8.1%. In congruence with Ighbaria’s (2013) 
and Sidek’s (2010) studies, the results show that the frequency of these questions is relatively high compared to 
the frequency of knowledge and application questions together. An example of the synthesis questions is on page 
52, “What was the writer’s purpose in writing this editorial?” 

Accordingly, the researchers believe that the number of the questions that required HOTS in the textbook is quite 
acceptable when compared with the results of Igbaria’s (2013) and Riazi and Mosalaejad’s (2010) studies, in 
which more emphasis were on questions that needed LOTS. 

4.2 Conclusions 

1) Master Class textbook incorporated reading comprehension questions that require both the lower level and the 
higher level cognitive demands. The textbook author, however, was not successful in varying the cognitive 
levels of the reading comprehension questions, where she exaggerated in presenting comprehension questions 
and nearly neglected application questions. 

2) The rising number of the questions that require high cognitive demands in the textbook suggests that the 
author considered the importance of training students on these levels of questions, which will eventually 
contribute to an effective EFL reading content especially at the university level or even for daily life needs. 

3) In line with EFL revised curriculum requirements, the textbook author succeeded in increasing the number of 
questions that require HOTS. 

4.3 Recommendations 

There is no doubt that the results of the study are encouraging. However, the English teachers and book authors 
have to take the following limitations of the study into consideration: the researchers analyzed Master Class 
textbook only, which is one of the textbooks intended for high proficiency level school students. Moreover, the 
study used Bloom’s taxonomy for analyzing the cognitive levels of the questions in the textbook Master Class. 

Accordingly the following recommendations are proposed by the researchers: 

1) Teachers have to evaluate any textbook before using it. The textbook should suit the students’ proficiency and 
cognitive level and meets the goals of the curriculum. 
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2) A good textbook can be an important agent to implement the change in the curriculum, but it cannot work 
alone. The teacher should decide what to take from the textbook and what to add by using other sources. They 
should also prepare well in order to direct the textbook content in a way that serves his/her goals and not vice 
versa. 

3) English textbooks should include reading tasks of various cognitive demands in order to equip students with 
the ability to perform tasks at any cognitive level in English at the school and university level. 

4) Textbooks authors should be aware of the different cognitive levels of the students in each grade level, they 
should accordingly use texts that respond to the students needs and present questions that encourage them to 
think and participate. 

5) Textbook authors should take the changes in the curriculum into consideration when writing new textbooks. 
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